Rank Selection in Low-rank Matrix Approximations: A Study of Cross-Validation for NMFs

We consider the problem of model selection in unsupervised statistical learning techniques based on low-rank matrix approximations. While k-fold crossvalidation (CV) has become the standard method of choice for model selection in supervised learning techniques, its adaptation to unsupervised matrix approximation settings has not received sufficient attention in the literature. In this paper, we emphasize the natural link between cross-validating matrix approximations and the task of matrix completion from partially observed entries. In particular, we focus on Non-negative Matrix Factorizations and propose scalable adaptations of Weighted NMF algorithms to efficiently implement cross-validation procedures for different choices of holdout patterns and sizes. Empirical observations on text modeling problems involving large, sparse document-term matrices suggest that these procedures enable easier and more accurate selection of rank (i.e., number of “topics” in text) than other alternatives for implementing CV.